Assessing the Impact of Cache Injection on Parallel Application Performance

The memory wall [13], the continuing disparity between processor and memory speeds, adversely affects the performance of many applications [8], particularly data intensive computations [11]. Cache injection addresses this disparity by placing incoming network data into a processor’s cache directly from the I/O bus. The effectiveness of this technique on application performance is dependent on several factors including the ratio of processor to memory speed, the NIC’s injection policy, and the application’s communication characteristics. In this work, I show that the performance of cache injection is directly proportional to the ratio of processor to memory speed, i.e., the higher the memory wall, the higher the performance. This result suggests that cache injection can be particularly effective on multi-core architectures (increasing number of cores compared to available channels to memory). Unlike previous work (focused on reducing memory copies incurred by the network stack) [1, 5], I show that cache injection improves application performance by leveraging the application’s temporal and spatial locality in accessing incoming network data. In addition, the application’s communication characteristics are key to cache injection performance. For example, cache injection can improve the performance of certain collective operations by 20%. Cache injection [1, 5, 3] is one of several techniques to alleviate the memory wall [9, 10, 8]. This technique reduces data latency and memory pressure (the number of requests issued to the memory controller per unit of time) by placing incoming network data directly into the cache. In current architectures, incoming network data is written to the system’s main memory and cached copies are invalidated. Cache injection replaces invalidate operations with updates if the corresponding cache lines are present, or allocate operations if they are not. In the next section, I describe how cache injection compares to a widely-used technique to reduce data latency, namely data prefetching. Unlike prefetching, cache injection significantly reduces memory pressure for I/O. Prefetching is driven by the access patterns of the processor (consumer of data), while cache injection is driven by the NIC (producer). This producer-initiated model makes cache injection prone to cache pollution. In Section 3, I show an example of this problem, and describe injection policies that determine what and when to inject into the cache to minimize pollution. In Section 4, I characterize application sensitivity to cache injection. In particular, I show that the performance of this technique is dependent on the degree to which systems are affected by the memory wall, the injection policy, and the application’s communication characteristics. Finally, I conclude in Section 5.